Supporting Interaction in the ROBOCARE Intelligent Assistive Environment

The ROBOCARE project is aimed at creating an instance of integrated environment with software and robotic agents to actively assist an elderly person at home. In particular the project has synthesized a proactive environment for continuous daily activity monitoring in which an autonomous robot acts as a main interactor with the person. This paper accounts for the combination of features that create interactive capabilities for personalized and mixed-initiative interaction with the assisted person. A project with an intelligent assistant The use of intelligent technology for supporting elderly people at home has been addressed in various research projects in the last years (Pollack et al. 2003; Pineau et al. 2003; Haigh et al. 2003; Pollack 2005). Increasing attention is also given to the synthesis of Cognitive Systems to produce aids that enhance human cognition capabilities. As an example, the project CALO (Myers 2006; Myers & Yorke-Smith 2005) has as its primary goal the development of cognitive systems able to reason, learn from experience, be told what to do, explain what they are doing, reflect on their experience, and respond robustly to contingencies. Other projects somehow connected to different aspects of this research topic are CMRADAR (Modi et al. 2004), whose aim is to develop an intelligent scheduler assistant, and CAMEO (Rybski et al. 2004) whose main objective is to build a physical awareness system to be used by an agent-based electronic assistant. All these projects have required the orchestration of different intelligent software technologies and have highlighted a number of important issues that need to be addressed: the problem of coordinating the distributed components as well as the problem of providing intelligent interaction with the user, are undoubtedly among the most critical. The ROBOCARE project shares several of the challenges with the above mentioned projects. Indeed ROBOCARE’s main motivations can be summarized as follows1: “The objective of this project is to build a distributed multi-agent system which provides assistance services Copyright c © 2007, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Quote from the original project proposal. for elderly users at home. The agents are a highly heterogeneous collection of fixed and mobile robotic, sensory and problem solving components. The project is centered on obtaining a virtual community of human and artificial agents who cooperate in the continuous management of an enclosed environment.” The project has involved research groups with different background with the goal of investigating how state of the art AI techniques could be combined to create new homeservice integration for elderly people (Cesta et al. 2003; Cesta & Pecora 2006; Bahadori et al. 2004). As a target domain we have chosen a prototypical home environment where the presence of an intelligent assistant would be of concrete help in the daily life of a cognitively impaired user through the integrated performance of advanced distributed components. The most important capability of an intelligent assistant is the continuous maintenance of a high level of situation awareness. This objective is obtained through the interaction of a number of intelligent physical and/or software agents: among others, vision-based sensors, which ensure the acquisition of continuously updated data from the environment; a schedule management software agent, which analyzes the status of every activity being performed within the monitored space; a mobile robotic platform able to behave robustly and continuously in the environment. The ultimate goal of the overall system is to provide cognitive support both on-demand, by guaranteeing a real-time question&answer service situated to the contextual knowledge of an assisted person, and proactively, by providing an event-driven support again grounded on what is going on in a daily living environment. Our efforts to integrate an autonomous robot in the intelligent environment have immediately made clear that the latter might naturally play the role of attention focalizer and main interactor with the user. Therefore we have concentrated most of the interaction capabilities in the robot, and, additionally, have chosen verbal communication as the main interaction modality. This paper specifically concerns the aspects related to the system’s context-awareness as well as interaction capabilities. In particular we describe how the constraintbased scheduling technology is used to maintain a knowledge repository aimed at supporting on-demand specific in(a) The “silent observer” (b) The current robotic assistant (c) The “proactive interactor” Figure 1: Evolving ROBOCARE demonstrators. teractions as well as enabling autonomous system initiative. The chosen domain and the addressed problem After having studied features of different physical environments for elderly people2, the home environment has been selected to develop the fielded integrated demonstrator. This choice is supported not only by the aim of improving home technology personalization, but also by recent studies, e.g., (Giuliani, Scopelliti, & Fornara 2005), that underscore the relevance of the attachment of elderly people to their home and the beneficial effects of increasing their independence at home. Figure 1(a) shows our first step toward the creation of an effective combined support service. We integrated sensors that monitor users’ activities3 with a schedule management environment called T-REX, (Tool for schedule Representation and EXecution (Pecora et al. 2006)), through which it is possible to represent of a set of activities and their quantitative temporal connections (i.e., a schedule of activities that the user is expected to carry out). The broad idea is to allow the specification and then the execution monitoring of a set of activities that the person usually performs or needs to perform due to prescription (on suggestion from his personal doctor for example). T-REX integrates a constraint-based scheduler (Cesta et al. 2001) with additional features for knowledge engineering, in particular for problem modeling. Particular attention has been given to the definition of “user-oriented terminologies” in order to easily synthesize both the basic elements of a target domain as well as different problem inSee (Cesta & Pecora 2006) for a broad presentation of the diversified project initiatives. In particular we used stereo cameras, endowed with specific software for people locatization and tracking (Bahadori et al. 2005). stances in the particular domain (i.e., in the form of activities and constraints among activities). For example, in the ROBOCARE context, T-REX allows the definition of “home domain activities” like breakfast, lunch, go-for-walk, and also temporal/causal links among activities like meal-boundmedication to express rules like “aspirin cannot be taken before eating”. Through this high level language an external user (a doctor, a relative of the assisted person, etc.) may define a network of activities, a schedule, that the observed person is supposed to carry out in the home environment during the day. Such a schedule is dispatched for execution and monitored using the underlying schedule execution technology. Information coming from the sensors is used for maintaining an updated representation of what is really happening in the environment. Even if human activity recognition4 is outside the scope of the project, it is worth highlighting how the sequence of observations from the artificial vision sensors allows to follow the evolution of the activities of the observed person (e.g., if and when she took a pill, when she had lunch, etc.). Based on the synthesis of these observations, the system is able to generate a report for the external users that underscores when the person’s activities have been performed within “reasonable” temporal boundaries or when important anomalies or even violations on their execution have been detected (Cesta et al. 2005; Bahadori et al. 2004). In this light the demonstrator in Figure 1(a), is a basic example of home Activity Monitor5 grounded on scheduling technology. Notice that, on its own, the domestic activity monitor acts as a “silent observer” and does not take initiative with respect to the elder person in any way. In this paper, we show how its indications can be There is plenty of recent research on activity recognition with sensors, e.g., (Pentney et al. 2006), that could potentially impact this class of applications. Further details on the Activity Monitor will be presented later